Model - Checking : Benchmarking and Techniques for Buffer Overflow Analysis by Kelvin Ku A thesis submitted in conformity with the requirements

Software Model-Checking: Benchmarking and Techniques for Buffer Overflow Analysis Kelvin Ku Master of Science Graduate Department of Computer Science University of Toronto 2008 Software model-checking based on abstraction-refinement has recently achieved widespread success in verifying critical properties of real-world device drivers. We believe this success can be replicated for the problem of buffer overflow detection. This thesis presents two projects which contribute to this objective. First, it discusses the design and construction of a buffer overflow benchmark for software model-checkers. The benchmark consists of 298 code fragments of varying complexity capturing 22 buffer overflow vulnerabilities in 12 open source applications. We give a preliminary evaluation of the benchmark using the SatAbs model checker. Second, the thesis describes the implementation of several components for supporting buffer overflow analysis in the YASM software model-checker.